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ABSTRACT 

According  to  National  Highway  Traffic  Safety  Administration  [NHTSA],  driver  inattention/distraction  is  one  of 
the  major  causes  of  road  accidents.  It  would,  therefore,  be  both  cost  and  safety  beneficial  if  a  distraction  detection  system 
could  be  developed.  This  paper  describes  a  real-time  non-intrusive  method  for  detecting  distraction  of  driver.  It  uses 
webcam  to  acquire  video  images  of  the  driver.  Visual  features  like  mouth  &  head  pose  which  are  typically  characterizing 
the  distraction  of  the  driver  are  extracted  with  the  help  of  image  processing  techniques  to  detect  types  of  distraction.  A 
study  about  the  performance  of  this  proposal  &  some  results  are  presented. 
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INTRODUCTION 

The  driving  task  is  a  complex  one,  requiring  a  substantial  degree  of  attention  and  concentration.  Despite  these 
complexities,  it  is  not  unusual  to  see  drivers  engaged  in  various  other  activities  while  driving,  such  as  in-vehicle  interaction 
with  passenger,  using  mobile,  eating/drinking,  looking  at  things  outside  the  vehicle,  etc.  These  activities  distracting  a  driver 
&  leads  to  danger  crashes.  It  would,  therefore,  be  both  cost  and  safety  beneficial  if  a  distraction  detection  system  could  be 
developed. 

DISTRACTION 

Driver  distraction  is  defined  as  a  specific  type  of  inattention  that  occurs  when  drivers  divert  their  attention  away 
from  their  primary  task  of  driving  to  focus  on  another  activity,  which  reduces  the  driver's  awareness,  decision-making, 
and/or  performance,  leading  to  an  increased  risk  of  corrective  actions,  near-crashes,  or  crashes. 

TYPES  OF  DISTRACTION 

There  are  four  types  of  driver  distraction  as  follows. 

•  Visual:  Occurs  when  a  driver  looking  at  something  other  than  road  such  as  looking  an  objects/events. 

•  Auditory:  Occurs  when  a  driver  hearing/talking  something  not  related  to  driving. 

•  Physical/Manual:  Occurs  when  a  driver  is  doing  something  physical  that  is  not  related  to  driving. 

•  Cognitive:  Occurs  when  a  driver  thinking  about  something  other  than  driving. 


Table  1:  Different  Sources  of  Distraction  &  its  Type 


Sr.  No. 

Activity/Source 

Types  of  Distraction 

1 

In-vehicle  interaction  with  persons  (like  baby, 
child,  adult,  passenger)  &  pets  in  adjacent/rear  seat 

Visual,  Auditory,  Manual, 
sometimes  Cognitive 

2 

Looking  at  things  outside  the  vehicle  (such  as 
persons,  animals,  building,  accident,  direction 
boards,  maps,  advertisements,  any  other 
events/objects) 

Visual,  Cognitive 
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Table  1:  Contd., 


3 

Eating,  drinking,  spilling,  preparing  to  eat/drink 

Visual,  Manual 

4 

I  Tci n (t/Hi ill i n tr  mnnilp  nhnnp   tf^vt  mf^cc^i  tri ntr 
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accessing  internet 

\/ 1  on  £1 1    Ai  i  Hi  toi*\7   \/I   m  ml 

V  IMUll.  r^ULlllAJl  V,  lVld.llLld.1, 

Cognitive 

5 

Adjusting  climate  control  (air  conditioner/heating), 
vehicle  control  (windows,  door  locks,  mirrors, 
seat,  steering  wheel,  seat  belt,  radio,  CD/DVD 
player),  storing/retrieving  items 

Visual,  Manual 

6 

Smoking  (includes  lighting  &  extinguishing), 
applying  makeup,  combing/fixing  hair,  other 
personal  hygiene 

Visual,  Manual 

7 

Driver  related  factors  (lost  in  thoughts,  day 
dreaming,  emotional-depression,  angry,  disturbed) 

Cognitive 

System  Overview 

The  complete  block  diagram  representation  of  the  proposed  system  is  as  shown  in  Figure  1.  In  literature,  there 
were  many  distraction  detection  methods  appearing  &  the  best  is  visual  feature  based  approach,  as  human  face  contained 
much  information  about  physical  condition.  Here,  mouth  &  head  tracking  done  in  real  time  using  webcams  to  detect 
different  types  of  distraction  of  driver. 
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Figure  1:  Complete  Block  Diagram  of  Proposed  System 


Image  Acquisition  Method 


The  proposed  system  used  iBall  face2face  CHD12.0  webcam  having  5G  wide  angle  lens,  6  LEDs  for  night  vision, 
with  brightness  controller,  interpolated  12M  pixels  still  image  &  2M  pixel  video  resolution.  The  camera  is  placed  in  front 
of  the  user,  approximately  30  cm  away  from  the  face.  The  camera  must  be  positioned  such  that  the  following  criteria  are 
met:  First  is  the  user's  face  takes  up  the  majority  of  the  image  &  second  is  the  user's  face  is  approximately  in  the  centre  of 
the  image.  After  that  converting  it  into  gray  scale,  as  RGB  components  are  dependent  on  the  lighting  conditions  thus  the 
face  detection  may  fail  if  the  lighting  condition  changes.  Next  is  face-detection  step,  the  face  region  is  found  within  the 
driver's  entire  facial  image  to  remove  unnecessary  background. 

Distraction  Decision 

The  continuous  distraction  estimate  needs  to  be  mapped  to  a  decision  whether  the  driver  is  distracted  or  not.  One 
approach  is  to  require  that  the  driver  is  looking  forward  for  some  minimum  time  before  he  or  she  is  considered  to  be 
attentive  but  there  will  be  possibilities  that  same  time  driver  may  be  talking  on  mobile  or  talking  with  passenger,  eating  or 
drinking  something  etc.  means  detecting  auditory  &  manual  distraction  with  monitoring  mouth  area  of  driver.  The  other 
approach  is  that  instead  of  looking  forward,  driver  is  looking  at  things  outside  from  the  vehicle  windows  (such  as  persons, 
animals,  building,  accident,  direction  boards,  maps,  advertisements,  any  other  events/objects)  or  may  be  looking  occupant 
in  adjacent  seat  (like  baby,  child,  adult,  passenger,  pet)  means  detecting  visual  distraction  with  monitoring  head  pose  of 
driver. 
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Figure  2:  Main  GUI  of  Proposed  System 


Inhibition  Criteria 

A  distraction  detection  method  determines  whether  a  driver  is  distracted  or  not,  but  when  and  in  which  way  the 
driver  will  be  warned  for  distraction  is  determined  by  a  warning  strategy.  There  are  situations  when  it  is  not  suitable  to  give 
distraction  warnings.  For  instance,  if  the  driver  is  braking  hard  he  or  she  is  probably  aware  of  the  situation  and  should  not 
be  disturbed  by  a  warning.  For  this  reason,  certain  criteria  can  be  set  up  to  inhibit  warnings.  Common  criteria  include: 

•  Direction  Indicators:  Changing  lanes  and  turning  means  that  the  driver  should  look  over  the  shoulder. 

•  Reverse  Gear:  Reverse  engaged  means  that  the  driver  should  look  over  the  shoulder. 

•  Brake  Pedal:  No  warning  should  be  given  while  driver  is  braking,  in  order  not  to  interfere  with  critical  driving 
manoeuvres. 

•  Steering  Wheel  Angle:  No  warning  should  be  given  while  the  driver  is  engaged  in  substantial  changes  of 
direction,  in  order  not  to  interfere  with  critical  driving  manoeuvres. 

•  Lateral  Acceleration:  No  warning  should  be  given  when  the  vehicle  makes  strong  movements,  in  order  not  to 
interfere  with  critical  driving  manoeuvres. 

Distraction  Detection  Method 

The  proposed  system  uses  Background  Subtraction  technique  for  detecting  distraction  of  driver.  The  basic  scheme 
of  background  subtraction  is  to  subtract  the  image  from  a  reference  image  that  models  the  background  scene.  If  the  pixel 
difference  is  greater  than  the  set  threshold,  then  determines  that  the  pixels  appear  in  the  moving  object,  otherwise,  as  the 
background  pixels.  As  the  camera  is  fixed,  the  background  model  can  remain  relatively  stable  in  the  long  period  of  time. 
Using  this  method  can  effectively  avoid  the  unexpected  phenomenon  of  the  Background,  such  as  the  sudden  appearance  of 
something  in  the  background  which  is  not  included  in  the  original  background.  Moreover  by  the  update  of  pixel  gray  value 
of  the  background,  the  impact  brought  by  light,  weather  and  other  changes  in  the  external  environment  can  be  effectively 
adapted.  Typically,  the  basic  steps  of  the  algorithm  are  as  follows: 

•  Background  modeling  constructs  a  reference  image  representing  the  background. 

•  Threshold  selection  determines  appropriate  threshold  values  used  in  the  subtraction  operation  to  obtain  a  desired 
detection  rate. 

•  Subtraction  operation  or  pixel  classification  classifies  the  type  of  a  given  pixel,  i.e.,  the  pixel  is  the  part  of 
background  (including  ordinary  background  and  shaded  background),  or  it  is  a  moving  object. 
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EXPERIMENTAL  RESULTS 
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Figure  3:  (a)  Main  GUI  Window  for  Auditory  Distraction,  (b)  Auditory  Distraction  is  Detected, 
(c)  Distraction  Result  for  Different  Frames  &  Mouth  Area 


I 


|0 


Fslflr.Sdloi  UltlMHIOi 


Fraiw  No  ;M 


ni|Nhii»kii  iii  i  i  ■ 


frame  Wo  :f  6 


DistMdiwt  deltdion 


(a) 


(b) 


Oljinacllw  Oelfttmn  tystem  sloped 

I     ^-  1 


piifiW«'>l 


(c) 


Cfetr»ctioii  Detection  jyjlan  sloped 

f      iu"  I 


(d) 


f  gr  r«  i 

'  'wr"1 
Frame  Ko  :i  M 


Real  Time  System  for  Detecting  Distraction  of  Driver 


89 


PisSractitwi  Detection  f ytfcm  st&jwl 


■5i>n  ] 

(e) 

Figure  4:  (a)  Main  GUI  Window  for  Visual  Distraction,  (b)  Visual  Distraction  is  Detected,  (c)  (d)  (e)  Distraction 
Detection  System  Stopped  when  Right  Indicator,  Left  Indicator  &  Reverse  Gear  are  Engaged 

CONCLUSIONS 

The  proposed  system  detecting  auditory,  manual  &  visual  distraction  of  driver  by  continuously  monitoring  mouth 
area  &  head  pose.  This  non-intrusive  approach  to  detecting  driver  distraction  without  interference  in  both  daytime  & 
nighttime  as  webcam  having  6  LEDs  for  night  vision,  with  brightness  controller.  However,  there  will  be  some  false 
detection,  where  the  results  are  not  good  when  there  is  quick  head-movement.  So,  future  work  will  be  done  based  on 
drivers  quick  head-movement  and  make  it  feasible  to  detect  driver's  distraction. 
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